The invention relates to a method and apparatus pertaining to dynamic transmission lines which allows multiple parameters, that control smart operations of dynamic transmission lines, to self-adapt simultaneously according to novel filter functions of the input to the transmission line and the unfiltered response of the target element. This formula or apparatus allows an unlimited number of smart transmission lines to be incorporated into networks of interacting elements.
Networks of interacting elements must be connected by transmission lines. Normally a great deal of effort is made to ensure faithful transmission from one element to the next, and many learning algorithms relate to strengthening or weakening the transmitted signals as a means of controlling the interaction between the elements. However, transmission lines may not always be inert in their transmission; that is, they may actually interact with the transmitted information, thereby modifying their transmission capability and hence the information transmitted. In the brain, for example, neurons are thought to communicate via such complex xe2x80x9csmart connectionsxe2x80x9d.
Such an active role of the transmission line could vastly amplify the information processing capability of a network of interacting elements since the transmission line carries out a xe2x80x9csmart operationxe2x80x9d by selecting to transmit only a subset of information. However, if a network contains millions or billions of such xe2x80x9csmart connectionsxe2x80x9d (hereinafter called smart transmission devices or STDs), it is essentially impossible to set the properties of each STD so that all the xe2x80x9csmart operationsxe2x80x9d carried out by the STDs in the network are compatible with each other. The present invention solves this apparent impossibility of orchestrating smart operations of a vast number of STDs.
The invention is directed to a process and apparatus for converting STDs into multiparameter self-adapting transmission lines (hereinafter called saSTD""s). STDs allow smart filtering of information transmitted from one element to another within a network of interacting elements, such as a xe2x80x9cneural networkxe2x80x9d and a multiple parameter supra-algorithm (hereinafter called MapSA) allows for control of the smart operation. MapSA serves to align the functions of even billions of STDs, allowing each one to transmit information compatible to the other. Such orchestration of a vast number of smart operations in a network could form the basis of a new generation of robots, computers, software programs, electronic circuits and network communication lines.
In order for information (represented by a series of binary electrical pulses) to be transmitted along a communication line from one brain cell to the next, the information must pass through a transfer station called a xe2x80x9csynapsexe2x80x9d. Synapses are essentially electrochemical transducers that convert a binary electrical signal (a pulse of set amplitude) into an analog chemical signal which, in turn, produces an analog electrical signal (a pulse of variable amplitude) in the target cell. About 50 years ago, it was suggested that these synapses can change their transmission strength and that such changes may form the basis of memory storage in neural networks (See D. Hebb, xe2x80x9cThe Organization of Behavior,xe2x80x9d J. Wiley and Sons, New York, 1949). In other words, Hebb originally declared that the strength of transmission lines (also referred to as the xe2x80x9cweightxe2x80x9d or xe2x80x9cgainxe2x80x9d of synaptic transmission) is a plastic parameter. Many different algorithms have been formulated in the past to alter the value of the synaptic gain which are published and form the basis of patents (see for example, Alkon et al., U.S. Pat. No. 5,402,522).
Recently is was declared that the non-linear transfer of information between neurons can also change (H. Markram and M. Tsodyks, 1996; Nature, 382, 807-810). The non-linear transmission of information across synapses is due to a delicate sequence of utilizing the gain of the synapse (gain is equivalent to all available xe2x80x9cresourcesxe2x80x9d of the synapse). For example, a synapse may begin using only a small fraction of its resources and then facilitate progressively increasing use of resources as each pulse of information arrives at the synapse. In addition, every time the resources are spent by the synapse, it takes time to replenish them, which leads to depression of transmission at a time when the information traffic is high.
The interplay between utilization, facilitation and depression results in rapid changes in transmission capacity from one moment to the next. The result is that the information that arrives in the synapse is not faithfully transmitted, but actually filtered in a complex non-linear manner; hence the term xe2x80x9csmart operationxe2x80x9d. Utilization, facilitation and depression are then three crucial parameters of STDs which determine their smart operation. These parameters are also referred to as xe2x80x9ckinetic parametersxe2x80x9d. NOTE: non-linear transmission does not refer to non-linear summation and complex transfer functions that determine how a single cell adds up the multiple analog information; rather, the invention relates to non-linearities in the communication lines (STDs) not in the elements of a network, which are usually seen as the processing elements (PEs).
The major difficulty in connecting PEs in a network via STDs is how to set the values of kinetic parameters that together determine the smart operation of a vast number of STDs. Using the MapSA, the STD is controlled with a device that first assesses the importance of the contribution of each kinetic parameter before determining how the parameter setting should change. The MapSA is different from any prior art in that functions are provided to evaluate the role of the parameters themselves in the transmission as compared to each other (i.e. in a relative manner) and, based on the results of the evaluation, the settings of the parameters are changed. In prior systems, the contribution of the entire synaptic output to the response of the target neuron formed the basis of adjusting the gain of the transmission.
The inventor""s studies of the functional significance of such dynamics indicate that MapSA could align any number of smart operations carried out by STDs across a network; a hitherto unimaginable and unsolvable task since the transmission of as many as 1014 STDs may need to be aligned. In other words, the information transmitted by each STD in the entire meshwork would become compatible and complementary with that transmitted by every other STD, allowing transmission of a multitude of small fragments of related information and spontaneous coagulation into a coherent and complete representation of an image, sound, smell etc., and therefore represent the most efficient form of information processing possible in networks.
An object of the invention is to provide a process and apparatus to construct saSTDs which can then be incorporated into networks of interacting elements. Since MapSA solves the problem of using STDs in networks this invention allows for a new generation of networks with virtually limitless capacity for change.
Other objects, advantages and novel features of the present invention will become apparent from the following detailed description of the invention when considered in conjunction with the accompanying drawings.